619 research outputs found
Multi-Task Predict-then-Optimize
The predict-then-optimize framework arises in a wide variety of applications
where the unknown cost coefficients of an optimization problem are first
predicted based on contextual features and then used to solve the problem. In
this work, we extend the predict-then-optimize framework to a multi-task
setting: contextual features must be used to predict cost coefficients of
multiple optimization problems, possibly with different feasible regions,
simultaneously. For instance, in a vehicle dispatch/routing application,
features such as time-of-day, traffic, and weather must be used to predict
travel times on the edges of a road network for multiple traveling salesperson
problems that span different target locations and multiple s-t shortest path
problems with different source-target pairs. We propose a set of methods for
this setting, with the most sophisticated one drawing on advances in multi-task
deep learning that enable information sharing between tasks for improved
learning, particularly in the small-data regime. Our experiments demonstrate
that multi-task predict-then-optimize methods provide good tradeoffs in
performance among different tasks, particularly with less training data and
more tasks
Walkability Optimization: Formulations, Algorithms, and a Case Study of Toronto
The concept of walkable urban development has gained increased attention due
to its public health, economic, and environmental sustainability benefits.
Unfortunately, land zoning and historic under-investment have resulted in
spatial inequality in walkability and social inequality among residents. We
tackle the problem of Walkability Optimization through the lens of
combinatorial optimization. The task is to select locations in which additional
amenities (e.g., grocery stores, schools, restaurants) can be allocated to
improve resident access via walking while taking into account existing
amenities and providing multiple options (e.g., for restaurants). To this end,
we derive Mixed-Integer Linear Programming (MILP) and Constraint Programming
(CP) models. Moreover, we show that the problem's objective function is
submodular in special cases, which motivates an efficient greedy heuristic. We
conduct a case study on 31 underserved neighborhoods in the City of Toronto,
Canada. MILP finds the best solutions in most scenarios but does not scale well
with network size. The greedy algorithm scales well and finds near-optimal
solutions. Our empirical evaluation shows that neighbourhoods with low
walkability have a great potential for transformation into pedestrian-friendly
neighbourhoods by strategically placing new amenities. Allocating 3 additional
grocery stores, schools, and restaurants can improve the "WalkScore" by more
than 50 points (on a scale of 100) for 4 neighbourhoods and reduce the walking
distances to amenities for 75% of all residential locations to 10 minutes for
all amenity types. Our code and paper appendix are available at
https://github.com/khalil-research/walkability
LEO: Learning Efficient Orderings for Multiobjective Binary Decision Diagrams
Approaches based on Binary decision diagrams (BDDs) have recently achieved
state-of-the-art results for multiobjective integer programming problems. The
variable ordering used in constructing BDDs can have a significant impact on
their size and on the quality of bounds derived from relaxed or restricted BDDs
for single-objective optimization problems. We first showcase a similar impact
of variable ordering on the Pareto frontier (PF) enumeration time for the
multiobjective knapsack problem, suggesting the need for deriving variable
ordering methods that improve the scalability of the multiobjective BDD
approach. To that end, we derive a novel parameter configuration space based on
variable scoring functions which are linear in a small set of interpretable and
easy-to-compute variable features. We show how the configuration space can be
efficiently explored using black-box optimization, circumventing the curse of
dimensionality (in the number of variables and objectives), and finding good
orderings that reduce the PF enumeration time. However, black-box optimization
approaches incur a computational overhead that outweighs the reduction in time
due to good variable ordering. To alleviate this issue, we propose LEO, a
supervised learning approach for finding efficient variable orderings that
reduce the enumeration time. Experiments on benchmark sets from the knapsack
problem with 3-7 objectives and up to 80 variables show that LEO is ~30-300%
and ~10-200% faster at PF enumeration than common ordering strategies and
algorithm configuration. Our code and instances are available at
https://github.com/khalil-research/leo
Machine Learning for Cutting Planes in Integer Programming: A Survey
We survey recent work on machine learning (ML) techniques for selecting
cutting planes (or cuts) in mixed-integer linear programming (MILP). Despite
the availability of various classes of cuts, the task of choosing a set of cuts
to add to the linear programming (LP) relaxation at a given node of the
branch-and-bound (B&B) tree has defied both formal and heuristic solutions to
date. ML offers a promising approach for improving the cut selection process by
using data to identify promising cuts that accelerate the solution of MILP
instances. This paper presents an overview of the topic, highlighting recent
advances in the literature, common approaches to data collection, evaluation,
and ML model architectures. We analyze the empirical results in the literature
in an attempt to quantify the progress that has been made and conclude by
suggesting avenues for future research.Comment: Accepted in IJCAI 2023 Survey Trac
Fast Matrix Multiplication Without Tears: A Constraint Programming Approach
It is known that the multiplication of an matrix with an matrix can be performed using fewer multiplications than what the
naive approach suggests. The most famous instance of this is Strassen's
algorithm for multiplying two matrices in 7 instead of 8
multiplications. This gives rise to the constraint satisfaction problem of fast
matrix multiplication, where a set of multiplication terms must be
chosen and combined such that they satisfy correctness constraints on the
output matrix. Despite its highly combinatorial nature, this problem has not
been exhaustively examined from that perspective, as evidenced for example by
the recent deep reinforcement learning approach of AlphaTensor. In this work,
we propose a simple yet novel Constraint Programming approach to find
non-commutative algorithms for fast matrix multiplication or provide proof of
infeasibility otherwise. We propose a set of symmetry-breaking constraints and
valid inequalities that are particularly helpful in proving infeasibility. On
the feasible side, we find that exploiting solver performance variability in
conjunction with a sparsity-based problem decomposition enables finding
solutions for larger (feasible) instances of fast matrix multiplication. Our
experimental results using CP Optimizer demonstrate that we can find fast
matrix multiplication algorithms for matrices up to in a short
amount of time
Neur2RO: Neural Two-Stage Robust Optimization
Robust optimization provides a mathematical framework for modeling and
solving decision-making problems under worst-case uncertainty. This work
addresses two-stage robust optimization (2RO) problems (also called adjustable
robust optimization), wherein first-stage and second-stage decisions are made
before and after uncertainty is realized, respectively. This results in a
nested min-max-min optimization problem which is extremely challenging
computationally, especially when the decisions are discrete. We propose
Neur2RO, an efficient machine learning-driven instantiation of
column-and-constraint generation (CCG), a classical iterative algorithm for
2RO. Specifically, we learn to estimate the value function of the second-stage
problem via a novel neural network architecture that is easy to optimize over
by design. Embedding our neural network into CCG yields high-quality solutions
quickly as evidenced by experiments on two 2RO benchmarks, knapsack and capital
budgeting. For knapsack, Neur2RO finds solutions that are within roughly
of the best-known values in a few seconds compared to the three hours of the
state-of-the-art exact branch-and-price algorithm; for larger and more complex
instances, Neur2RO finds even better solutions. For capital budgeting, Neur2RO
outperforms three variants of the -adaptability algorithm, particularly on
the largest instances, with a 5 to 10-fold reduction in solution time. Our code
and data are available at https://github.com/khalil-research/Neur2RO
LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations
Can a Large Language Model (LLM) solve simple abstract reasoning problems? We
explore this broad question through a systematic analysis of GPT on the
Abstraction and Reasoning Corpus (ARC), a representative benchmark of abstract
reasoning ability from limited examples in which solutions require some "core
knowledge" of concepts such as objects, goal states, counting, and basic
geometry. GPT-4 solves only 13/50 of the most straightforward ARC tasks when
using textual encodings for their two-dimensional input-output grids. Our
failure analysis reveals that GPT-4's capacity to identify objects and reason
about them is significantly influenced by the sequential nature of the text
that represents an object within a text encoding of a task. To test this
hypothesis, we design a new benchmark, the 1D-ARC, which consists of
one-dimensional (array-like) tasks that are more conducive to GPT-based
reasoning, and where it indeed performs better than on the (2D) ARC. To
alleviate this issue, we propose an object-based representation that is
obtained through an external tool, resulting in nearly doubling the performance
on solved ARC tasks and near-perfect scores on the easier 1D-ARC. Although the
state-of-the-art GPT-4 is unable to "reason" perfectly within non-language
domains such as the 1D-ARC or a simple ARC subset, our study reveals that the
use of object-based representations can significantly improve its reasoning
ability. Visualizations, GPT logs, and data are available at
https://khalil-research.github.io/LLM4ARC.Comment: 17 pages, 11 figure
- …